Background: There have been several reports from DoD users about DoD certificate issues recently. The root of the issue is associate with DoD Root certificates chaining improperly to the (Federal Government) Common Policy Root Certificate Authority (CA). When this occurs on DoD systems, PKI validation does not work properly and may result in any of the following: 1. DoD user denied access to DoD web sites. 2. DoD users are unable to access DREN VPNs. 3. DoD signed emails in outlook appear invalid. 4. DoD users experience extensive delays with Outlook or Internet Explorer during validation. 5. DoD users receive a prompt to install the Common Policy Root CA when opening a signed email of a DoD sender whose workstation is misconfigured. . Issue: On November 23, 2009, Microsoft released an update KB931125 which updated the third-party certification authorities (CAs) that are trusted by Microsoft and whose root certificates are distributed via the Windows Root Certificate Program. The following certificates are the root cause: Thumbprint: dc92f91bab283472023b32178504e19bf7d9a94c Issuer/Subject: Iss: OU=Entrust, OU=FBCA Sub: OU=Entrust, OU=FBCA Thumbprint: 0a0e46657f4148df2d1c6778ea9308a8ca41989f Issuer/Subject: Iss: CN=Common Policy, OU=FBCA Sub: OU=Entrust, OU=FBCA Thumbprint: 3bae7b920ee6616755be4fa287777eef2f6b33f6 Issuer/Subject: Iss: CN=Common Policy, OU=FBCA Sub: OU=Entrust, OU=FBCA Thumbprint: dc92f91bab283472023b32178504e19bf7d9a94c Issuer/Subject: Iss: OU=Entrust, OU=FBCA Sub: CN=DoD Interoperability Root CA 1 Thumbprint: eea68fc8701e41e6429a341ae4162bbda634f7f4 Issuer/Subject: Iss: CN=DoD Interoperability Root CA 1 Sub: CN=DoD Root CA 2 Steps Taken so far: Step 1: DoD Administrators with workstations affected by this issue should decline all KB931125 updates in Windows Server Update Services 3.0. From the WSUS 3.0 manager console, search for " 931125 ". Right click the update and select Decline for all updates related to 931125. Step 2: DoD Administrators with workstations affected by this issue should remove the certificates below either with the DoD PKE tool or some other means. The DoD PKE "FBCA Cross-Certificate Removal Tool" is available from the following location: PKE on DKO: https://www.us.army.mil/suite/page/474113 under "Downloads" The following cross-certificates should be removed from the Local Computer and User Intermediate Certification Authority store: (DISA PKE Support) Common Policy à Entrust (FBCA) cross-certificate Subject: OU=FBCA, OU=FBCA,O=U.S. Government,C=us Issuer: OU=Common Policy,OU=FBCA,O=U.S. Government,C=us Serial # 18 cc d6 6b 00 01 00 00 00 6f Valid To: Thursday, April 23, 2015 9:20:26 AM Common Policy à Entrust (FBCA) cross-certificate (Revoked) Subject: OU=FBCA, OU=FBCA,O=U.S. Government,C=us Issuer: OU=Common Policy,OU=FBCA,O=U.S. Government,C=us Serial # 62 fa 21 6f 00 01 00 00 00 56 Valid To: Friday, March 21, 2014 12:25:49 PM Entrust (FBCA) à IRCA cross-certificate Subject: CN=DoD Interoperability Root CA 1,OU=PKI,OU=DoD,O=U.S. Government,C=US Issuer: OU=Entrust,OU=FBCA,O=U.S. Government,C=US Serial # 45 1d e5 23 Valid To: Friday, December 31, 2010 12:00:00 AM IRCA à DoD Root CA 2 cross-certificate Subject: CN=DoD Root CA 2, OU=PKI,OU=DoD,O=U.S. Government,C=US Issuer: CN=DoD Interoperability Root CA 1,OU=PKI,OU=DoD,O=U.S. Government,C=US Serial # 0C Valid To: Thursday, March 03, 2011 10:22:43 AM The following self-signed certificate(s) should be removed from the Local Computer and User Trusted Root Certification Authority store: Entrust (FBCA) self-signed certificate Subject: OU=Entrust,OU=FBCA,O=U.S. Government,C=US Issuer: OU=Entrust,OU=FBCA,O=U.S. Government,C=US Serial # 45 1d e1 72 (DISA PKE Support) For Domain Administrators , the tool can run on domain workstations using a Group Policy Startup script. First, a hidden share should be configured on a server to capture logs that the script was run. The hidden share should allow everyone to write to the share, checking both NTFS and Share permissions. In this example, the share was called "FBCA-log$". Create another shared folder called "FBCA_Crosscert_fix" with read-only permissions for all users. Next, create a new Group Policy Object named " Configuration: Remove CommonPolicy CA Script", the group policy should be applied to the entire domain to ensure the misconfigured Root CA certs are completely removed from the domain. Next, create a new Startup script named " FBCA-scriptremove.cmd " with the following scripts. @echo off IF EXIST \\servername\FBCA-log$\%computername%.txt goto end \\servername\FBCA_Crosscert_fix\FBCA_crosscert_remover_v104.exe /commonpolicy /s > \\servername\FBCA-log$\%computername%.txt :end Next, download the DoD PKE Cert Removal tool from: https://www.us.army.mil/suite/page/474113 and place FBCA_crosscert_remover_v104.exe in the \\servername\FBCA_Crosscert_fix\ share. Lastly, ensure that the " Turn off Automatic Root Certificates Update" setting is "Enabled" in Group Policy on the domain. This setting can be set under: Computer Configuration | Administrative Templates | System | Internet Communication Management | Internet Communication settings. Step 2a: Additional Requirement for Windows XP SP2 workstations In some cases, it has been reported the " Turn off Automatic Root Certificates Update" GPO setting does not stop Windows from updating the Root Certificates automatically. Two additional steps must be taken. First, the " FBCA-scriptremove.cmd " script must be modified to include code to uninstall the " Update Root Certificates " Windows component installed under Windows XP SP2 by default . The script should now look like this: The script should now look like this: @echo off IF EXIST \\servername\FBCA-log$\%computername%.txt goto end \\servername\FBCA_Crosscert_fix\FBCA_crosscert_remover_v104.exe /commonpolicy /s > \\servername\FBCA-log$\%computername%.txt :end IF EXIST \\ servername \FBCA-log$\%computername%_WXP-03-Uninst-Updt-Rt-Cert-Comp.txt goto end sysocmgr /i:%windir%\inf\sysoc.inf /u:\\servername\software\FBCA_Crosscert_fix\rootautoupdateuninstall.txt /q > \\ servername \FBCA-log$\%computername%_WXP-03-Uninst-Updt-Rt-Cert-Comp.txt :end In addition, the following text file named " rootautoupdateuninstall.txt " should be created, located in the same share that was created to store the FBCA_crosscert_remover_v104.exe utility. For example: \\servername\FBCA_Crosscert_fix\ " rootautoupdateuninstall.txt " should contain the following text. [components] rootautoupdate = off Validation: If everything is setup and working correctly, when hosts restart, they will run the script one time, and write two log files to the \\servername\FBCA-log$\ share. The first file will be named after the host. The file will contain a log of the results of the FBCA_crosscert_remover_v104.exe utility. The log will contain one of two outcomes. No certs were found: ##### FBCA cross-certificate removal tool ##### Created by DoD PKE Engineering January 2010 BACKGROUND: The DOD PKI PMO has discovered problems building certificate chains when the DoD Root and Intermediate certificates have not been properly installed. PROBLEM: Administrators must install the DoD Root and latest intermediate CA certificates on all workstations and servers. SYMPTOMS: Users can experience delays when performing operations with DoD PKI issued certificates. Inconsistent path building can occur where DoD PKI certificates chain up to the Common Policy root or cross-certificates mentioned below. CERTIFICATES: Thumbprint Issuer/Subject dc92f91bab283472023b32178504e19bf7d9a94c Iss: OU=Entrust, OU=FBCA Sub: OU=Entrust, OU=FBCA 0a0e46657f4148df2d1c6778ea9308a8ca41989f Iss: CN=Common Policy, OU=FBCA Sub: OU=Entrust, OU=FBCA 3bae7b920ee6616755be4fa287777eef2f6b33f6 Iss: CN=Common Policy, OU=FBCA Sub: OU=Entrust, OU=FBCA dc92f91bab283472023b32178504e19bf7d9a94c Iss: OU=Entrust, OU=FBCA Sub: CN=DoD Interoperability Root CA 1 eea68fc8701e41e6429a341ae4162bbda634f7f4 Iss: CN=DoD Interoperability Root CA 1 Sub: CN=DoD Root CA 2 RESOLUTION: * Disable the Microsoft Root Update Service (DISA STIG requirement) * Install the DoD Root and Intermediate CA certificates which can be done by running the latest version of InstallRoot (as an administrator). InstallRoot can be obtained from https://www.dodpke.com/installroot/ * Run this tool. DEPENDENCIES: * Microsoft Windows 2000 SP3 or newer Operating System * .NET Framework 2.0 or above USAGE: /COMMONPOLICY Additionally removes the Common Policy self-signed certificates /S Silent mode - doesn't require user to hit <ENTER>. NOTE: Administrative privileges are required to remove certificates from the LocalMachine store. Starting to search for certificates... Searching the LocalMachine Root certificate store...certificates not found. Searching the LocalMachine CA certificate store...certificates not found. Searching the LocalMachine My certificate store...certificates not found. Searching the CurrentUser Root certificate store...certificates not found. Searching the CurrentUser CA certificate store...certificates not found. Searching the CurrentUser My certificate store...certificates not found. Finished. Certs were found and removed: ##### FBCA cross-certificate removal tool ##### Created by DoD PKE Engineering January 2010 BACKGROUND: The DOD PKI PMO has discovered problems building certificate chains when the DoD Root and Intermediate certificates have not been properly installed. PROBLEM: Administrators must install the DoD Root and latest intermediate CA certificates on all workstations and servers. SYMPTOMS: Users can experience delays when performing operations with DoD PKI issued certificates. Inconsistent path building can occur where DoD PKI certificates chain up to the Common Policy root or cross-certificates mentioned below. CERTIFICATES: Thumbprint Issuer/Subject dc92f91bab283472023b32178504e19bf7d9a94c Iss: OU=Entrust, OU=FBCA Sub: OU=Entrust, OU=FBCA 0a0e46657f4148df2d1c6778ea9308a8ca41989f Iss: CN=Common Policy, OU=FBCA Sub: OU=Entrust, OU=FBCA 3bae7b920ee6616755be4fa287777eef2f6b33f6 Iss: CN=Common Policy, OU=FBCA Sub: OU=Entrust, OU=FBCA dc92f91bab283472023b32178504e19bf7d9a94c Iss: OU=Entrust, OU=FBCA Sub: CN=DoD Interoperability Root CA 1 eea68fc8701e41e6429a341ae4162bbda634f7f4 Iss: CN=DoD Interoperability Root CA 1 Sub: CN=DoD Root CA 2 RESOLUTION: * Disable the Microsoft Root Update Service (DISA STIG requirement) * Install the DoD Root and Intermediate CA certificates which can be done by running the latest version of InstallRoot (as an administrator). InstallRoot can be obtained from https://www.dodpke.com/installroot/ * Run this tool. DEPENDENCIES: * Microsoft Windows 2000 SP3 or newer Operating System * .NET Framework 2.0 or above USAGE: /COMMONPOLICY Additionally removes the Common Policy self-signed certificates /S Silent mode - doesn't require user to hit <ENTER>. NOTE: Administrative privileges are required to remove certificates from the LocalMachine store. Starting to search for certificates... Searching the LocalMachine Root certificate store... * FOUND Certificate: CN=Common Policy, OU=FBCA, O=U.S. Government, C=us Issuer : CN=Common Policy, OU=FBCA, O=U.S. Government, C=us Thumbprint : CB44A097857C45FA187ED952086CB9841F2D51B5 Serial: 293647AAE38AAC864A2356F2CAB761AF - Removing Certificate...SUCCESSFUL * FOUND Certificate: CN=Common Policy, OU=FBCA, O=U.S. Government, C=us Issuer : CN=Common Policy, OU=FBCA, O=U.S. Government, C=us Thumbprint : 76B76096DD145629AC7585D37063C1BC47861C8B Serial: 39E3815404C50AB247EFFEF336CFC698 - Removing Certificate...SUCCESSFUL Searching the LocalMachine CA certificate store...certificates not found. Searching the LocalMachine My certificate store...certificates not found. Searching the CurrentUser Root certificate store...certificates not found. Searching the CurrentUser CA certificate store...certificates not found. Searching the CurrentUser My certificate store...certificates not found. Finished. The second log file will be named after the host followed by " _WXP-03-Uninst-Updt-Rt-Cert-Comp.txt " If the host is running Windows XP or Server 2003, the Windows Component will be removed, and the following information will be logged. If the host is running Vista or above, the file will be blank. 401.339.0: 0x80070002 (WIN32: 2) 401.432.0: 0x80070002 (WIN32: 2) 401.1246.0: 0x80070002 (WIN32: 2): C:\WINDOWS\CAPolicy.inf 104.685.0: 0xe0000102 (INF: -536870654): Components 110.379.0: 0xe0000102 (INF: -536870654) 104.1282.0: 0xe0000102 (INF: -536870654) 104.2445.0: 0xe0000102 (INF: -536870654) 104.2436.0: 0xe0000102 (INF: -536870654) 104.2482.0: 0xe0000102 (INF: -536870654) 104.2482.0: 0xe0000102 (INF: -536870654) 104.2482.0: 0xe0000102 (INF: -536870654) 104.2587.0: 0xe0000102 (INF: -536870654) 104.2587.0: 0xe0000102 (INF: -536870654) 104.2587.0: 0xe0000102 (INF: -536870654) 104.2502.0: 0xe0000102 (INF: -536870654) 104.2502.0: 0xe0000102 (INF: -536870654) 104.2502.0: 0xe0000102 (INF: -536870654) The script will run one time, if the log files it creates remain present for subsequent logons. The script will not run if the domain is unavailable at boot. During logon when the script is working, the startup may take longer than usual. This should be a onetime occurrence to repair this issue. Problem: DisableRootAutoUpdate set to Enabled by doesn't seem to work for this case. Users are still finding their systems are being updated automatically with the "Common Policy Root CA" certificate.

First of all, this is not the proper place to post this information as it is an infosec violation and opsec violation. Additionally, DISA doesnt know how to handle the Federal Bridge as they have 0 experience with it. The AutoUpdate for the Root certificates is ONLY for Microsoft certificates that is included with the OS for those that go through the MS process to be added to the certificate stores. The reason you are getting the certificates in your store is because of Outlook 2007. In Vista they made a hotfix available to resolve the issue however as far as Office goes, they did not make one for XP. So when someone with those certificates in their stores send an email, it sends the entire chain to the recipients. In Windows 7, the fix was built-in the OS so the OS and Office didnt do this. Originally, this was noticed in Vista as it caused all kinds of issues. This is because Vista crawls the AIA and SIA extensions to build the cert store and its CRL repository for its built-in OCSP functionality. Office would just take the certificates found in Vista and pass them along. So if the recipent was an XP user with Office 2007, they would get it in their store and pass it on later when they emailed someone. Now, if you are a person that does business with other government entities, you will need those certificates to function with them. The only people I am aware of that do not actually support the Federal Bridge is DISA. The proper fix would be for DISA to add the appropriate missing certs to their store as it would not allow anyone other than DoD to access the systems anyways because of the path direction for that particular leg of the FBCA. So to make a long story a little shorter, the issue will always be present until you upgrade to Windows 7. Then those certs will not effect you.

First of all, this is not the proper place to post this information as it is an infosec violation and opsec violation. Additionally, DISA doesnt know how to handle the Federal Bridge as they have 0 experience with it. The AutoUpdate for the Root certificates is ONLY for Microsoft certificates that is included with the OS for those that go through the MS process to be added to the certificate stores. The reason you are getting the certificates in your store is because of Outlook 2007. In Vista they made a hotfix available to resolve the issue however as far as Office goes, they did not make one for XP. So when someone with those certificates in their stores send an email, it sends the entire chain to the recipients. In Windows 7, the fix was built-in the OS so the OS and Office didnt do this. Originally, this was noticed in Vista as it caused all kinds of issues. This is because Vista crawls the AIA and SIA extensions to build the cert store and its CRL repository for its built-in OCSP functionality. Office would just take the certificates found in Vista and pass them along. So if the recipent was an XP user with Office 2007, they would get it in their store and pass it on later when they emailed someone. Now, if you are a person that does business with other government entities, you will need those certificates to function with them. The only people I am aware of that do not actually support the Federal Bridge is DISA. The proper fix would be for DISA to add the appropriate missing certs to their store as it would not allow anyone other than DoD to access the systems anyways because of the path direction for that particular leg of the FBCA. So to make a long story a little shorter, the issue will always be present until you upgrade to Windows 7. Then those certs will not effect you. ...come on down from your horse there buddy... what exact rules are being violated? Doesn't your post also violate them? ;) I don't see how a solution will be found without giving out the details of the problem. These certificates were pushed out by Microsoft in an update from November 2009. It has also been affecting Windows 7 machines with Office 2007 in our environment. I've also seen these certs effect/break Macs too, so... it's not just a Vista problem. About the only thing I've seen positive results with is adding these "bad certs" to the untrusted store... but that's not really a great solution. There should be a way to block them completely from being auto installed automatically, without user knowledge. It seems like that's kind of a bigger picture problem on Microsoft's end. There doesn't even seem to be a way to see when they are installed in the Event logs. Does anyone know how to turn on verbose logging of the Certificate Stores, to see these auto installs?

Rob, I'm working on the same problem. It seems you guys are further than I am on this but I have contact info you can have for PKI people.

The OPSEC and INFOSEC is part of your annual training required for having your account and feel free to contact your local IA shop for further guidence. The specific message you posted Rob contains FOUO and was not intended to be released outside of DoD. That was your only violation. If you would of left that out, it wouldnt of been an issue. In case your wondering my post didnt contain any FOUO so the answer to your question Rob is no. I work Information Assurance and I am a Senior PKI Engineer. I also have been working the Federal Bridge arena for 2 years now. We dont have the issues you are having on Vista or Windows 7 but then we knew what to do as it is our subject matter. That isnt a cheap shot, its just that not everyone has access to PKI SME's to assist them. As far as Vista and Windows 7, you need SP2 for Vista. The certs do still show up if you receive them from an email but the system doesnt go out and fetch them on their own like it did before. Common signs of the issue is 2-5 minutes to get to any SSL site when web browsing an HTTPS site, email taking up to five minutes to send if it is signed, and logons taking 30 minutes or more. With SP2 those are resolved. The only thing left is someone who sends it because it was already in the other persons store and thus it was included in the email. For example external users, customers, etc. On Windows 7 there is a Group Policy that shuts off SIA and AIA extension crawling. It will also allow you to get to sites with those certs in your store as it will not prevent you from getting to the sites. That is because it will not look at the AIA and SIA extensions to chain the CA's together. Those certs have been going around since 2008. Mindy Tidwell (Microsoft Federal Services employee) published the issue on her FDCC blog. However, you may like to know that CoreStreet (company that does OCSP and SCVP systems that does revocation checks) do publish regular updates that also updates your cert store. They do publish the Federal Bridge certs. Additionally, the FDCC images of XP and Vista included the same certs in their cert stores. You can verify that by going to the NIST FDCC site and looking at their change/history information on their images. The Actual fix would be for the sites you have issues with, to actually add them to their trust store. The chain involved is DoD's side of the chain so their is no ill effect by doing so and it would not grant external access to the systems. Lastly, add those certs to the OCSP and or SCVP systems so revocation checks are faster. That will resolve the issue. Being that DoD PKE has asked for my help with the Federal Bridge and have actually directed other entities to contact who I work for if they need assistance with the Federal Bridge, I think I earned my position on the horse.

CAPI2 Eventlogs can be set to enabled for Vista and Windows 7 which would log everything that goes through the CAPI.

Windows 7 Group Policy Public Key Policies - Certificate Path Validation Settings - Network Retrieval Tab - Check box to define policy and leave the rest unchecked. It wont fix XP with Office 2007 but I do know it works on Windows 7. The KB specified didnt install on any Windows 7 machines that I am aware of outside of the government. At least none of my machine at home or my friends and family machines. But then again from my understanding, most people had problems with it installing anyways until MS killed the update with a newer one.

I'd love some POCs. Thanks!

I'm still not sure what FOUO information was posted, I wrote the bulk of my post from my head. Minus the text from the DISA PKI Removal Tool... but anyway, it sounds like you do deserve to be on your horse. :D I'd love to talk to you about this issue offline, to get a better understanding of what the deal is, and how to fix it. Thanks!

add @msn.com to my id and you can contact me there. I can not answer questions from work as it is a conflict of interest since you are not my customer. But as I have time, I will answer questions for you. Just keep in mind that was is sent to this address is available to anyone that gains access to the servers. So use caution when emailing to that address. To ensure it doesnt get filtered as SPAM, put PKI in the subject.

Go here: http://iase.disa.mil/pki-pke/function_pages/tools.html Use the FBCA tool this way: - Install the DoD roots as admin - Run the FBCA tool as admin - Run the FBCA tool as the user Very easy to implement as a login script and through SCCM as an update to do thousands of machines.

The problem with the tool is you are not compliant with the OMB mandate to work with the Federal Bridge and accept their credentials. The idea behind it all is to know who you are doing business with regardless of what government entity you are dealing with. Also, being able to validate their certs for signed documents as well as be able to encrypt sensitive email. By using that tool, you are putting those certs in your untrusted certificate store and essentially, revoking their certs. Thus if the user(s) actually needs those certs for doing business with a Federal Bridge member, it will fail and the user(s) will not be able to do business with them. Therefore, use of the tool should be with extreme caution and selectively at that. The other issue, at times, is the Install Root putting the Interoperability cert in your store. But as I said before, if they just put the other certs in their store and used OCSP or SCVP it wouldn't be an issue. The only time it would, is if the cert path was the opposite direction. For example: DoD Root CA -> DoD Interoperability CA -> Federal Root CA -> Federal Bridge Member Root CA -> Federal Bridge User Issuing CA -> End Entity cert (names are not exact intentionally). In this situation, there would be an issue if you are only requiring they have a cert from your trusted store. However, access should not be based on just having a cert in your trusted store. But with the configuration I specified in previous posts, you shouldn't have that specific chain problem as it wont add those certs in the means stated here. It would only build to the Anchored Root for our path and send that. Additionally, how certs are being used is starting to change as there is a move towards Federated Identities as part of the Identity Access Management (which is where the Federal Identity & Credential Access Management Roadmap comes into play).